Goto

Collaborating Authors

 alternative use


Do LLMs Agree on the Creativity Evaluation of Alternative Uses?

Rabeyah, Abdullah Al, Góes, Fabrício, Volpe, Marco, Medeiros, Talles

arXiv.org Artificial Intelligence

This paper investigates whether large language models (LLMs) show agreement in assessing creativity in responses to the Alternative Uses Test (AUT). While LLMs are increasingly used to evaluate creative content, previous studies have primarily focused on a single model assessing responses generated by the same model or humans. This paper explores whether LLMs can impartially and accurately evaluate creativity in outputs generated by both themselves and other models. Using an oracle benchmark set of AUT responses, categorized by creativity level (common, creative, and highly creative), we experiment with four state-of-the-art LLMs evaluating these outputs. We test both scoring and ranking methods and employ two evaluation settings (comprehensive and segmented) to examine if LLMs agree on the creativity evaluation of alternative uses. Results reveal high inter-model agreement, with Spearman correlations averaging above 0.7 across models and reaching over 0.77 with respect to the oracle, indicating a high level of agreement and validating the reliability of LLMs in creativity assessment of alternative uses. Notably, models do not favour their own responses, instead they provide similar creativity assessment scores or rankings for alternative uses generated by other models. These findings suggest that LLMs exhibit impartiality and high alignment in creativity evaluation, offering promising implications for their use in automated creativity assessment.


Pizza, porn and whale snot: seven alternative uses for drones

The Guardian

News that a British Airways plane was hit by a drone before landing safely at Heathrow airport has once again highlighted how drones can be a nuisance and, potentially, dangerous. We all know about the military uses of drones (bomb lots of people, surveillance), and how drones can be used for nefarious purposes (theft, voyeurism), but there are actually some pretty cool uses for drones too. Last year the Ocean Alliance partnered with tech heads Yuneec to create "snot bots"; drones with petri dishes attached. For research purposes, the drones are flown over water to catch spray and snot from whales when the animals exhale. There is already a drone journalism lab at the University of Nebraska-Lincoln and the University of Missouri also offers a drone journalism course.